Skip to content

Send mask as binary data#123

Draft
cfs-data wants to merge 12 commits intomainfrom
feature/send-mask-as-binary
Draft

Send mask as binary data#123
cfs-data wants to merge 12 commits intomainfrom
feature/send-mask-as-binary

Conversation

@cfs-data
Copy link
Collaborator

@cfs-data cfs-data commented Feb 5, 2026

This PR explores the option of sending masks as binary form data. The edit-scan endpoint now accepts a pydantic model with edit image parameters as JSON string, and a binary field for the mask data.

This made automatically generating Swagger documentation quite a bit harder. For this, the model_json_schema() method was overriden to include the examples and descriptions from BaseParameters and MaskParameters, and the openapi_extra field was used to inject the generated JSON documentation. To view the examples in the /docs playground, select either multipart/form-data or application/json from the drop-down menu in the edit-scan section (this is not the default unfortunately).

TODO's:

  • parse binary array data outside of the pydantic model
  • automatically generate Swagger documentation for Json() field using the pydantic model
  • remove all Depends
  • add unit tests to reach minimum coverage
  • add docstrings
  • rewrite logic to use run_pipeline()
  • add option for compression (e.g. bitpacked arrays)
  • [N] run benchmarks for different formats

@cfs-data cfs-data marked this pull request as draft February 5, 2026 15:50
@cfs-data cfs-data changed the title Send mask as binary Form data [Draft] Send mask as binary Form data Feb 5, 2026
@cfs-data cfs-data changed the title [Draft] Send mask as binary Form data [Draft] Send mask as binary data Feb 5, 2026
@cfs-data cfs-data changed the title [Draft] Send mask as binary data Send mask as binary data Feb 7, 2026
@github-actions
Copy link

github-actions bot commented Feb 7, 2026

Diff Coverage

Diff: origin/main..HEAD, staged and unstaged changes

  • src/models.py (100%)
  • src/preprocessors/pipelines.py (100%)
  • src/preprocessors/router.py (40.0%): Missing lines 183-185,187,193-194
  • src/preprocessors/schemas.py (100%)

Summary

  • Total: 38 lines
  • Missing: 6 lines
  • Coverage: 84%

src/preprocessors/router.py

Lines 179-191

  179     Accepts an X3P scan file and edit parameters (mask, zoom, step sizes),
  180     validates the file format, parses it according to the parameters, and
  181     creates a vault directory for future outputs. Returns access URLs for the vault.
  182     """
! 183     if mask_data is not None:
! 184         if params.mask_parameters is None:
! 185             raise HTTPException(HTTPStatus.UNPROCESSABLE_CONTENT, "Invalid request: missing mask parameters.")
  186 
! 187         _ = parse_mask_pipeline(
  188             raw_data=await mask_data.read(),
  189             shape=params.mask_parameters.shape,
  190             is_bitpacked=params.mask_parameters.is_bitpacked,
  191         )

Lines 189-198

  189             shape=params.mask_parameters.shape,
  190             is_bitpacked=params.mask_parameters.is_bitpacked,
  191         )
  192 
! 193     _ = parse_scan_pipeline(params.scan_file, params.step_size_x, params.step_size_y)
! 194     vault = create_vault(params.tag)
  195 
  196     logger.info(f"Generated files saved to {vault}")
  197     return ProcessedDataAccess.generate_urls(vault.access_url)

@github-actions
Copy link

github-actions bot commented Feb 7, 2026

Code Coverage

Package Line Rate Branch Rate Health
. 96% 88%
comparators 100% 100%
computations 100% 100%
container_models 99% 100%
conversion 97% 86%
conversion.export 100% 100%
conversion.filter 92% 83%
conversion.leveling 100% 100%
conversion.leveling.solver 100% 75%
conversion.plots 98% 85%
conversion.preprocess_impression 99% 91%
conversion.preprocess_striation 89% 58%
extractors 98% 75%
mutations 100% 100%
parsers 98% 80%
parsers.patches 89% 60%
preprocessors 95% 71%
processors 100% 100%
renders 98% 50%
utils 91% 75%
Summary 96% (2033 / 2107) 81% (218 / 268)

Minimum allowed line rate is 50%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant